Speech-driven Animation with Meaningful Behaviors

Conversational agents (CAs) play an important role in human computer interaction. Creating believable movements for CAs is challenging, since the movements have to be meaningful and natural, reflecting the coupling between gestures and speech. Studies in the past have mainly relied on rule-based or data-driven approaches. Rule-based methods focus on creating meaningful behaviors conveying the underlying message, but the gestures cannot be easily synchronized with speech. Data-driven approaches, especially speech-driven models, can capture the relationship between speech and gestures. However, they create behaviors disregarding the meaning of the message. This study proposes to bridge the gap between these two approaches overcoming their limitations. The approach builds a dynamic Bayesian network (DBN), where a discrete variable is added to constrain the behaviors on the underlying constraint. The study implements and evaluates the approach with two constraints: discourse functions and prototypical behaviors. By constraining on the discourse functions (e.g., questions), the model learns the characteristic behaviors associated with a given discourse class learning the rules from the data. By constraining on prototypical behaviors (e.g., head nods), the approach can be embedded in a rule-based system as a behavior realizer creating trajectories that are timely synchronized with speech. The study proposes a DBN structure and a training approach that (1) models the cause-effect relationship between the constraint and the gestures, (2) initializes the state configuration models increasing the range of the generated behaviors, and (3) captures the differences in the behaviors across constraints by enforcing sparse transitions between shared and exclusive states per constraint. Objective and subjective evaluations demonstrate the benefits of the proposed approach over an unconstrained model.Comment: 13 pages, 12 figures, 5 table

MEGA: Multimodal Alignment Aggregation and Distillation For Cinematic Video Segmentation

Previous research has studied the task of segmenting cinematic videos into scenes and into narrative acts. However, these studies have overlooked the essential task of multimodal alignment and fusion for effectively and efficiently processing long-form videos (>60min). In this paper, we introduce Multimodal alignmEnt aGgregation and distillAtion (MEGA) for cinematic long-video segmentation. MEGA tackles the challenge by leveraging multiple media modalities. The method coarsely aligns inputs of variable lengths and different modalities with alignment positional encoding. To maintain temporal synchronization while reducing computation, we further introduce an enhanced bottleneck fusion layer which uses temporal alignment. Additionally, MEGA employs a novel contrastive loss to synchronize and transfer labels across modalities, enabling act segmentation from labeled synopsis sentences on video shots. Our experimental results show that MEGA outperforms state-of-the-art methods on MovieNet dataset for scene segmentation (with an Average Precision improvement of +1.19%) and on TRIPOD dataset for act segmentation (with a Total Agreement improvement of +5.51%)Comment: ICCV 2023 accepte

Predicting Medical Students’ Academic Burnout Based on Academic Self-Efficacy: The Mediating Role of Academic Grit

Introduction: Academic burnout has significant negative consequences for students. Accordingly, identifying inhibiting factors is of particular importance. The purpose of this research was to investigate the mediating role of academic grit in the relationship between the academic self-efficacy and academic burnout in medical students. Methods: The study was descriptive with a correlational design. The population consisted of all students of Kashan University of Medical Sciences, Iran, in academic years 2022-2023 among which 391 students were selected through the use of multistage sampling. The data collection instruments included Questionnaires of Midgley academic self-efficacy, Bresso et al.'s academic burnout and general grit of Duckworth and Quinn. The data were analyzed using Pearson's correlation coefficient and Structural Equation Modeling. Results: The results revealed that academic self-efficacy directly predicts academic grit (p<0.01, ß=0.63) and the direct effect of academic self-efficacy on students' academic burnout is negative and significant. (p<0.01, β=0.31). Besides, academic self-efficacy could indirectly affect academic burnout through academic grit. (p<0.01, ß=0.29). This way, it can be inferred that academic grit plays a mediating role in the relationship between academic self-efficacy and academic burnout. Conclusion: Strengthening students' academic self-efficacy can increase their academic stability and thus help to reduce their academic burnout. Therefore, it is suggested to pay attention to the improving of self-efficacy and academic grit in educational programs